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Abstract 

This research study examines preservice teachers’ experiences of learning 
through game design. In particular, we investigate how their perceptions of 
digital games have evolved through the process of designing and building their 
own educational games. We are also concerned with the knowledge and rea¬ 
soning skills that preservice teachers incorporate into their games. Further, we 
investigate the 21 st century skills involved in the game design and implemen¬ 
tation process. In this case study, the participants are first-year students at a 
university in Western Canada enrolled in a secondary mathematics methods 
course. A total of 21 preservice teachers (10 males and 11 females) in the class 
participated, ranging from ages 20 to 45 years old. We collected three sets of 
data: open-ended pre- and post-surveys, games created by preservice teachers, 
and follow-up interviews with selected participants. The analysis of the data 
shows that the game-building experience impacted preservice teachers’ per¬ 
ceptions related to challenges, problems solving, and attitudes toward gaming 
and design. Further, their games demonstrated these preservice teachers’fair 
understanding of pedagogical and cognitive components. (Keywords: digital 
game-based learning, enactivism, secondary mathematics, preservice teach¬ 
ers, learning by game design and building) 


Technology has had a drastic impact on society in the last few decades. 
From the prevalence of digital social networks to the ubiquitous cell phone, 
how we interact as citizens and how the global economy operates has 
changed. Citizens of this new world require a different set of skills to be 
productive and active members of society (Jenkins et al„ 2006). As a result, 
educators are being called upon to help students develop 21 st century skills. 
These 21 st century skills include learning and innovation skills; information, 
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media, and technology skills; and life and career skills (Partnership for 21 st 
Century Skills, 2004). 

How do educators prepare children for this new age? Digital games 
(hereafter games) are generating increasing interest (Gee, 2008a). This is 
not surprising, considering 97% of teens aged 12-17 play computer, Web, 
portable, or console games (Lenhart et al„ 2008). In particular, learning as 
digital game design and building is proposed to be an effective approach 
(Gee, 2008a, 2008b; Kafai et al., 2008). Because students are already inter¬ 
ested in games, educators can harness this to exercise 21 st century skills in 
their classrooms by having students design, develop, and build their own 
digital games. Game design can be used specifically in mathematics educa¬ 
tion, as it promotes creative thinking and problem-solving skills that can be 
transferred beyond programming to the classroom (Li, 2010). 

Helping students acquire 21 st century skills through digital game building 
and design demands teachers’ deep understanding of the related technol¬ 
ogy and the appropriate pedagogy. Immersing preservice teachers in the 
experience of designing and building their own games provides an effective 
opportunity to develop such technological and pedagogical expertise (King, 
2011). An opportune time to do this is during teacher training. Game design 
and building may help preservice teachers develop appropriate attitudes, 
pedagogy, and technical skills related to gaming and the role of gaming in 
the classroom. Further, the process can provide preservice teachers the op¬ 
portunity to experience and test the pedagogical and technical strategies to 
promote learning of 21 st century skills. 

In this report we examine preservice teachers’ experiences of learning 
through game design. In particular, we investigate how their perceptions of 
digital games have evolved through the process of designing and building 
their own educational games. We are also concerned with the knowledge 
and reasoning skills that preservice teachers (hereafter teachers) incorporate 
into their games. Further, we investigate the 21 st century skills involved in 
the game design and implementation process. 

Theoretical Framework 

This paper is grounded in the theory of “enactivism” (Li et al„ 2010) as it 
applies to educational technology. The theory of enactivism has two major 
premises. The first premise is that the mind, body, and world are inseparable 
(Fenwick, 2000). Enactivism believes that the personal history of the subject 
and the setting affect the outcomes of events. That is, the outcome of a spe¬ 
cific learning activity is determined by the environment of the activity (time, 
place, etc.) and by the participant (gender, cultural background, action, etc.) 
(Li et al, 2010). 

The second premise is that learning occurs through feedback within the 
system (Fenwick, 2000). As such, cognition is a complex process of individu¬ 
als interacting and affecting each other and their environments (Davis et al., 
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2008). Enactivism places emphasis on knowledge co-authoring. As research¬ 
ers (Li et al„ 2010) state, “ [a] 11 living systems have to be involved in cogni¬ 
tion, and cognition is active rather than passive” (p. 8). 

Enactivism holds different beliefs from both behaviorism and con¬ 
structivism. Behaviorism and constructivism share similar assumptions, 
although they appeared to be completely different (Davis & Sumara, 

2002). A noticeable shared assumption is that cognition sits inside the 
individual’s body, isolated from the world and other people. Reality (the 
real world) is separated from the mind, even though the two theories have 
different views about where it is. Behaviorists assert that reality is exter¬ 
nal to the knower, is structured, and that the structure can be modeled. 

The purpose of learning is to mirror this abstract reality and its structure 
through thinking (Davis & Sumara, 1997). Constructivism, regardless of 
the version, assumes that reality is formed from the mind of the knower 
through his/her construction (Dewey, 1933; Vygotsky, 1978). Therefore, 
both behaviorism and constructivism accepts dualism and interpret cogni¬ 
tion in mechanistic ways. 

Enactivism, in contrast, rejects dualism that divides self from world, 
mind from body, or subject from object. Therefore, both behaviorism and 
constructivism focus on knowledge, whereas enactivism emphasizes know¬ 
ing. Enactivism rejects the idea that knowledge consists of separate objects 
different from the world. Rather, “all cognition exists in the interstices of a 
complex ecology of organism relationality” (Davis & Sumara, 1997, p.l 10). 

In enactivism, thinking and cognition are grounded in action (Holton, 
2010). The core characteristics of digital game building are doing and co¬ 
authoring, which provide an ideal platform for creating an enactivist world 
in the context of a participatory culture. In this learning world, the teachers 
are asked to design and build a digital game targeting a specific mathematics 
topic. The assumption is that the games they create will embody the cogni¬ 
tive processes used and the processes intended for students to use when 
playing their games. 


Literature Review 

Today, students are plugging into, participating in, and authoring digital 
content on a daily basis. In the late 20 th century, digital games became a 
significant component of childrens culture (Gareau & Guo, 2009). As part 
of this culture, digital game play now has widespread appeal among chil¬ 
dren (Li, 2010). Instructional games appeal to students, as they provide a 
new learning culture that corresponds with students’ pre-existing habits and 
interests (Prensky, 2001). 

A participatory culture compels participants to discover new fact-finding, 
content, and technical skills, which enable them to enhance their new media 
literacy skills (Jenkins et al„ 2006). Burdick and Willis (2010) have posited 
that “literacy itself is situated, networked, and contingent... and continually 
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negotiated.... Design thinking ... specifically supports this constant emer¬ 
gence” (p. 3). Seymour Papert studied the link between student learning 
with technology, specifically a hands-on perspective through student design 
and construction (Papert, 1998). Since then, the concept of digital learning 
with technology and games has been explored more in depth (Hsaio, 2007; 
Kirriemuir &McFarlane, 2004). 

Digital game-based learning is a promising area of research and debate, 
as it promotes contextualized learning, creates and harnesses motivation in 
learning capacities, and encourages curiosity (Gee, 2008a; Kirriemuir & Mc- 
Farlane, 2004; Prensky, 2006). There is evidence to show that digital games 
are effective in promoting learning because students are able to interact with 
virtual worlds where players solve simulations of real-world problems and in 
the process learn real-world skills, knowledge, and values (Gee, 2008b). Ad¬ 
ditionally, game play is valuable in the realm of problem solving and learn¬ 
ing because it lowers the emotional stakes of failing. That is, students are 
able to learn through trial and error and suspend real-world consequences 
(Jenkins et al„ 2006). 

Designing and creating in this digital environment are aspects of gam¬ 
ing literacy (Zimmerman, 2007). They are embedded within the social skills 
necessary to interact within a larger community (Jenkins et al., 2006) and 
are criteria for the acquisition of 21 st century skills. Due to students’ ongo¬ 
ing relationship with technology, learning by game building has encouraged 
researchers to perform empirical studies in the subject areas of mathematics 
(Kafai & Ching, 1996; Noss & Hoyles, 2006), computer science (Korte et al., 
2007), and elementary science topics such as physics (Kafai et al., 1997; Li, 
2010). In these studies, researchers analyzed and interpreted students’ game¬ 
building experiences to find patterns, processes, and connections between 
building and learning. 

Proponents of learning by game building are looking at digital literacy 
skills fostered in teachers for the purpose of training them to facilitate 
burgeoning 21 st century skills among students (Becker, 2007; Gee, 2008b). 
Inservice teachers have difficulties with understanding how to use games in 
their classrooms and have concerns about time and availability of technol¬ 
ogy (Ertzberger, 2009). It has been found that teachers require a connection 
to the curriculum in order to integrate games (Ketelhut & Schifter, 2011). 

A proactive method proposed to keep up with the technological skills of 
students is to provide teacher training in technology and the use of digital 
games in the classroom (Becker, 2007; Tiong & Yong, 2008). 

A study by Kafai and colleagues (1998) explored preservice teachers’ ex¬ 
periences of designing games for mathematics learning. They found that the 
game design activities transformed these preservice teachers from extrinsic 
game designers (i.e., viewed game idea and math content as separate) to 
intrinsic game designers (i.e., viewed game idea and math content as inte¬ 
grated). Further, these teachers’ understanding of students’ thinking provid- 
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ed the foundations for their game design. During the game design process, 
these teachers played both the designer and player roles in an ongoing way. 
That is, they considered designing and playing concurrently. 

Attitudes toward technology predict how one will behave with technol¬ 
ogy. Specifically, a teacher’s personal experience with technology may not be 
sufficient to provide the confidence needed for him or her to design lessons 
that incorporate game-based technology (Sardone & Devlin-Scherer, 2010). 
Regardless of the educational benefits and prevalence of digital games within 
student culture, the majority of teachers do not see the immediate value 
of educational online gaming, gaming communities, or games as a com¬ 
munication tool, but rather see games as rewards. This perception is largely 
informed by prior experience with games (Schrader et al., 2006). 

Despite perceptions of games and game use, teachers are open to new 
applications of technology and consider games to be important educational 
tools (Schrader et al., 2006). However, many are not fully aware of the 
pedagogical uses and benefits of digital games. To combat this, preservice 
education should incorporate opportunities to experience and reflect upon 
game use (Schrader et al., 2006; Wright, 2009). Further, when teachers were 
immersed in a 21 st century learning environment, they developed new per¬ 
spectives regarding the use of games for instruction; certain games could be 
used to teach content and have students develop 21st century skills (Sardone 
& Devlin-Scherer, 2010). 

Earlier research had also explored teacher development of software, 
although not directly connecting to game design. In the early 1980s, a focus 
of educational computing was on teaching teachers to program so that 
they could create their own software. For example, Oates (1986) reported 
several different projects where professors developed their own programs 
for instructional purposes. The projects were conducted in the University 
of Illinois, University of Oklahoma, Pennsylvania State University, and the 
City University of New York, respectively. Other studies explored how K-12 
teachers could learn programming to improve their students’ learning of 
various subjects, such as English and mathematics (Jackson, 1983; Kagan, 
1989). This line of research, however, was questioned and criticized because 
teachers had neither the skill nor the time to make effective software (Brac- 
ey, 1986; Crull, 1987). 

Although research in the area of learning through game building is in 
its early stages, majority of the research has focused either on students or 
inservice teachers (Gee, 2008b; Kafai, 2006; Kafai et al, 2008). There has 
been very little done in the area of preservice teachers. Research, thus far, 
involving preservice teachers has been mainly from the viewpoint of game 
play—that is, how to incorporate premade digital games into lessons and 
how game play embodies 21 st century skills. 

Further, some research has been done in the area of preservice teachers’ 
perceptions of digital games (Sardone & Devlin-Scherer, 2010; Schrader et 
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al., 2006). Limited research investigates the process of preservice teachers 
designing and building their own games. This study intends to bridge this 
gap by investigating the experiences of preservice teachers as they designed 
and built their own digital games. It is important to realize that this line of 
inquiry is not following the failed design strategy of the 1980s. 

Research Questions 

The following questions guided our research investigation: 

• How did designing and building a game affect teacher perceptions about 
digital game building, if at all? 

• When teachers were designing games, what pedagogical areas and cogni¬ 
tive skills did they incorporate into their games? 

• What kinds of 21 st century skills were demonstrated in the teachers’ game 
design and building experience? 

In this paper, we used the 21 st century skills defined by the Partnership 
for 21 st Century Skills (2004). Such skills include creativity, critical thinking, 
problem solving, communication and collaboration, information literacy, 
media literacy and information, communications and technology literacy, 
flexibility, adaptability, initiative, self-direction, social and cross-cultural 
skills, productivity, accountability, leadership, and responsibility. 

Methods 

Adapting a case-study approach, this study investigated preservice teachers’ 
perceptions of digital games and their experiences designing and building an 
educational digital game. The case study fits well with our enactivist view¬ 
point because both share the goals “to place the researcher within the study 
so as to avoid objectification and to conduct research that is transformative” 
(Creswell, 1998, p. 83). 

Participants 

The participants of this study were first-year students at a university in west¬ 
ern Canada enrolled in a secondary mathematics methods course as part of 
their after-degree bachelors of education program. A total of 21 teachers (10 
males and 11 females) in the class participated. Their ages ranged from 20 to 
45 years old. Two of the researchers also participated in the study. 

The course focused on promoting understanding of mathematics educa¬ 
tion in the context of a learner-focused, inquiry-based, and field-oriented 
platform. One project was for teachers to work individually or in small 
groups to design and build a digital game that focused on algebra. 

Data and Instruments 

Three sets of data were collected: open-ended pre- and post-surveys, 
games created, and follow-up interviews with selected participants. We 
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Table 1 . Pre-Survey Questions 

The first survey was given in the first week of class. The questions were as follows: 

1. What do you think about designing/building your own games for students (e.g., benefit, disadvantage)? 

2. What do you think about kids creating their own education games for their learning (usually described as learning through 
designing)? Why? 

3. In what ways can you use the ‘learning as game-designing 1 in your classrooms (e.g., elementary music class, adult training 
program)? Please describe specifics (e.g. who, what, when, how). 

4. When designing your own games for students, what are some challenges you would expect? 


Table 2. Post-Survey Questions 

The second survey was given at the end of the semester after students had finished their digital games. The questions were as follows: 


1. What do you think about the experience of designing your own games (including paper prototyping and digital games) for 
students? 

2. What are things you’ve learned/benefit through this game-designing/building experience? Give examples/scenarios to 
explain your points. 

3. What are some challenges you find in designing your own games (including paper prototyping and digital games) for 
students? 

4. Do you plan to use the game you designed in your class? Why or why not? 

5. What do you think about kids creating their own education games for their learning? Why? 


administered two surveys, a pre-survey and a post-survey, to examine 
teachers’ perceptions about digital games (see Table 1 and Table 2). We 
administered the pre-survey during the first week and the post-survey at 
the end of the semester. 

The second set of data consisted of an evaluation of all participants’ 
created games using the pedagogical rubric and the cognitive rubric we 
developed. The pedagogical rubric focused on the educational aspects 
of the games, including three categories: knowledge, game play, and 
playability. The iterative development process started with a discussion 
among all the researchers about the curriculum standards (e.g., NCTM 
Standards [2000]) and the critical pedagogical understanding required 
for mathematics teachers. The initial discussions resulted in the identi¬ 
fication and definition of the following five categories: problem solving, 
representation, active learning, exploration and reasoning, and con¬ 
nections. Each researcher independently evaluated three representative 
games first using the rubric, and then all researchers worked together to 
discuss the results, paying particular attention to disagreements. Such 
discussions led to the inclusion of critical components, the clarification 
of definitions, and the modification of the ranking system. For example, 
we realized that games could be unfairly evaluated because of poor 
game design. We added engagement/motivation, participation, and user 
friendly and ease of play so as not to punish limited technical skills in 
the pedagogical areas. We developed a final version of the pedagogical 
rubric after many rounds of testing and fine-tuning each category. Each 
researcher then independently evaluated all the games first, and then the 
whole team discussed them until reaching a consensus. See Appendix A 
(pp. 333-335) for details. 
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The cognitive rubric, adapted from the revised Bloom’s Taxonomy (An¬ 
derson & Krathwohl, 2001), evaluates the depth of thinking skills that are 
required in playing the game. The development of this rubric took a similar 
iterative process. For this rubric (see Appendix B, pp. 336-337), the assump¬ 
tion was that the cognitive skills were nested within each other, meaning 
that if a higher-order thinking skill was used, then the lower-order cognitive 
skills were implied. For example, applying the cognitive skill of analyzing 
indicated the use of all the cognitive skills below it (e.g., remember, under¬ 
stand, apply). 

The third set of data consisted of the follow-up interviews focusing on 
the 21 st century skills with participants 6 weeks after the course finished. We 
invited everyone for a follow-up interview and interviewed six teachers on 
a volunteer basis. We showed these teachers the Partnership of the 21 st Cen¬ 
tury (2004) framework and then asked them to identify the skills they used 
and provide examples. 

We recognized the subjective nature of some subscales, especially engage¬ 
ment/motivation. To establish fidelity, we provided consistent training to the 
evaluators. We also tested inter-rater reliability. If the inter-rater reliability 
coefficient was lower than .78, another rater was added to evaluate the game 
until all four researchers assessed the game. If at this point the agreement 
rate was still below .78, we had a detailed discussion until we reached a con¬ 
sensus. In addition, as two researchers served the dual role of teacher and 
researcher, we made sure that they did not evaluate their own games. 

Instructional Process 

In this study, teachers were engaged in game design activities. The process 
started with an introduction of various games during a 2-hour session in 
which teachers played and evaluated free online games, such as Crayon 
Physics (http://www.crayonphysics.com). After an hour of play, each worked 
in a small group to create a lesson plan. The requirement was that the lesson 
had to use one of the games to teach an algebraic concept. As many teachers 
were nongamers, such activities enabled them to become familiar with this 
media. For teachers who were gamers, this exercise gave them an opportu¬ 
nity to re-examine digital games through the eyes of teachers, focusing on 
educational value rather than just the entertainment value of the games. 

A few weeks later, the teachers had another class (3 hours) focusing on 
game design and paper prototyping. At the beginning, teachers were asked 
to adapt their favorite movies and transform the movies into board games. 

In small groups, teachers came up with their game designs, paper prototyped 
their ideas, and then play-tested each others games. During this session, they 
did not use computers. While play-testing a game, everyone in the class was 
engaged either as an observer or as a player. The designers were asked to re¬ 
frain from talking or showing and to rather simply observe what players were 
doing. The instructor gave verbal feedback after each game was tested. 

316 I Journal of Research on Technology in Education | Volume 45 Number 4 
Copyright © 2013, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 



Preservice Teachers' Digital Game Design Expeience 



Shellshocked 2 


Equalia 



Fraction Mahjong The Multiplying Game 


Figure 1. Screenshots of teachers' games. 


At the middle of the semester, the teachers began to develop their own 
game ideas. The instructor introduced the Web app Scratch (http://scratch. 
mit.edu) in a class and allowed the teachers to play with the software for 
about 1 hour. The instructor also created an online forum for these teachers 
to discuss ideas, help others, and share resources. For example, the instruc¬ 
tor provided a link introducing another software program, Kudo (http://re- 
search.microsoft.com/en-us/projects/kodu/), in this forum and encouraged 
them to explore. The teachers had the freedom to use any software platform, 
but Scratch was the only one introduced in a face-to-face class. At the end of 
the semester, they spent 2 hours showcasing and playing each other’s games 
as a celebration. 

During this process, the instructor provided no instruction about how to 
choose an algebraic topic. They were also not required to research the use of 
games in teaching mathematics. 

Games Created 

A total of 14 games were created: 4 sandbox games, 1 platform game, 1 role- 
playing game, and 1 arcade game. The majority of the games (10 of the 14) 
were designed for late elementary or junior high students, while 3 games 
were for high school students. Table 3 (p. 318) provides a brief summary of 
the games that the teachers created. We describe four sample games in detail 
because they were referred to often in the subsequent sections (see Figure 1 
for images). 
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Table 3. Summary of Games Created 


Game Name 

Audience 

(grade) 

Program 

Type of game 

Main curriculum topic 

Pedagogical score 
(out of 18) 

Cognitive score 
(out of 6) 

5 points 4 stars 

5 to 9 

SMART 

Notebook 

Not a game 

Geometry 

8 

2 

Awesome 

Adventure 

7 

Little Big 
Planet 2 

Platform 

Fractions, decimals 
and percent 

10 

2.4 

Call of Math 

7 to 9 

iWeb 

Board game 

Variety of topics 

11 

3.2 

Castle Quest 

9 

RPG Maker 

VX 

Role playing 
game 

Solving and manipu¬ 
lating algebraic equa¬ 
tions and expressions 

4 

3 

Date Game 

9 to 10 

Scratch 

Board game 

Linear relationship 
applications 

10 

2.5 

Equalia 

6 to 7 

Kodu 

Sandbox 

Balancing equations 

6 

4 

Fraction 

Mahjong 

7 

SMART 

Notebook 

Board game 

Equivalent fractions 

7 

4 

Fraction Town 

7 to 9 

Kodu 

Sandbox 

Fractions 

6 

2.25 

Fraction War 

7 

Scratch 

Card game 

Fractions 

6 

2 

Kodu’s Treasure 

Hunt 

7 

Kodu 

Sandbox 

Linear equation ap¬ 
plications 

8 

4 

Linus the Lion's 

Quest for Stars 

10 to 12 

Scratch 

Board game 

Graphing 

10 

4 

Multiplying 

Game 

7 

Scratch 

Arcade 

Multiplying 

3 

1 

Shellshocked 2 

10 

Kodu 

Sandbox 

Systems of linear 
equations 

12 

6 

Who Wants 

to be a Math 

Millionaire? 

unknown 

Scratch 

Board game 

Linear equation ap¬ 
plications 

8 

1.6 


Shellshocked 2. In this sandbox game, the player decides between two 
story lines: They can either help Turtle Montana destroy the oil refinery that 
is making people sick or they can help the Cloud King by rescuing his kid¬ 
napped son. Either way, the players have to collect a certain number of fire 
and water crystals. To do this, they need to spend a certain amount of time 
in the fire area, where they gain fire crystals but lose water crystals. Alter¬ 
natively, they could spend a certain amount of time in the water area, where 
they gain water crystals but lose fire crystals. The amount of time spent in 
each area is not apparent from the outset and needs to be worked out by set¬ 
ting up and solving linear equations. Players have to work out the problem 
and the solution on their own. 

Fraction Mahjong. This is a modified Mahjong game. Instead of finding 
identical tiles, the players need to find tiles with equivalent fractions. Like 
Mahjong, the strategy is deciding which tiles to choose, opening up blocked 
tiles in such a way that all tiles can be removed. Many cognitive processes 
are used in this strategy, such as planning, hypothesizing, organizing, and 
testing. 
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Equalia. In this game, grade 6 and 7 players learn about balancing equa¬ 
tions by making an equal number of objects on two sides of a fence. Some¬ 
times mathematical objects are missing that need to be added. Sometimes 
extra objects need to be removed. 

The Multiplying Game. This remedial multiplication exercise for grade 7 
students asks players to multiply single-digit numbers to keep critters away 
from a breakdancer. The critters are pushed back when the player multiplies 
correctly; otherwise the critters continue to advance. As the game progress¬ 
es, the critters advance faster and faster. 

Data Analysis 

We answered the first research question by examining teacher responses 
to two surveys to identify salient themes. We then performed a frequency 
count for each theme. 

We answered the third research question through analysis of the games, 
classroom observations, and follow-up interviews. The thematic analysis 
for both research question 1 and 3 adapted a three-step process, starting 
with open coding (Corbin & Strauss, 1990) of all written work. We worked 
independently to identify recurring and salient themes during this step. In 
the second step, we compared these naturally emerging themes with the 
existing literature of learning-as-game designing. This allowed us to examine 
the summarized data and create general categories from the full data set. We 
created a matrix of merged themes to support the overall interpretation of 
the data. 

In the third phase, we conducted a cross-case analysis via matrices and 
other displays to further condense the data and draw comparisons (Miles & 
Huberman, 1994). We summarized the coded information about each par¬ 
ticipant in matrices and then compared them to cluster cases. Grouping the 
data under different codes allowed us to see different patterns and emergent 
themes. During this iterative process, we adapted a constant comparative 
approach (Corbin & Strauss, 1990) to create, refine, and recreate themes and 
codes (Miles & Huberman, 1994). 

We answered the second research question by evaluating the games cre¬ 
ated based on the pedagogical rubric and the cognitive rubric. We gave each 
category in the pedagogical rubric a value: 0 for did not meet expectations, 1 
for met expectations, and 2 for exceeded expectations. We obtained the total 
score for the game (out of 18) by summing up the ranks of each category. 
(See Table 4 and Table 5, p. 320.) 

For the cognitive rubric, we rated each problem in the game from 1 to 6, 
where 1 meant that solving the problem would require minimal cognitive 
skills and 6 meant that solving the problem would require extensive cogni¬ 
tive skills. We determined the cognitive score for the entire game by find¬ 
ing the average of the cognitive scores for each of the problems. We then 
analyzed the games by looking at the descriptive statistics of the results. The 
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Table 4. Summary of Pedagogical Rubric (Individual Categories) 


Category 

Did Not Meet 
Expectations 

Met 

Expectations 

Exceeded 

Expectations 

Mean 

Median 

Mode 

Problem solving 

7 

3 

4 

0.79 

0.5 

0 

Representation 

2 

10 

2 

1.00 

1 

1 

Active learning 

2 

9 

3 

1.07 

1 

1 

Exploration & Reasoning 

6 

6 

2 

0.71 

1 

0 

Connections 

9 

3 

2 

0.50 

0 

0 

Strategy 

7 

5 

2 

0.64 

0.5 

0 

Engagement/motivation 

0 

13 

1 

1.07 

1 

1 

Participation 

1 

10 

3 

1.14 

1 

1 

User friendly & ease of play 

3 

9 

2 

0.93 

1 

1 


Note. To create a score for each rubric, a "does not meet expectations” is a 0, a “meets expectations” is a 1, and an “exceed expec¬ 
tations” is a 2. The most a game can get is 18 points. 


Table 5. Summary of Pedagogical Rubric (Total Scores) 


Statistic 

Result 

Count 

14 

Mean 

7.9 

Median 

8 

Mode 

8 

Minimum 

3 

Maximum 

11 


data we obtained from both rubrics were nonordinal. The median was the 
measure of center that we used for analysis, and we applied no measures of 
variation. (See Table 6.) 


Results 


Perceptions 

How did the game design and building experience affect teachers’ percep¬ 
tions? The analysis of teachers’ survey showed that three major themes 
emerged: challenges, problem solving, and attitudes toward gaming and 
design. 

Challenges 

The first and foremost theme that emerged was challenges in game design 
and building, indicating that the game design experience changed teachers’ 
perceptions about specific challenges they would face. 

Before the teachers began the project, the greatest challenge that 62 percent 
envisioned was coming up with a game design that would be fun and interest¬ 
ing to play. They were concerned about their ability to create a game that would 
engage the students while still being educational, as these comments reflect: 
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Table 6. Summary of Cognitive Rubric (Total Scores) 


Statistic 

Result 

Count 

14 

Mean 

2.93 

Median 

2.45 

Mode 

4 

Minimum 

1 

Maximum 

6 


It takes a lot of effort creating a game that is fun, engaging and incorpo¬ 
rates learning concepts. (PH, pre-survey) 

[Y]ou can very easily design a game that students find boring, thereby 
defeating the purpose of using video games in the first place. (DC, pre¬ 
survey) 

Teachers were also concerned about having enough creativity to come up 
with an interesting game design. More than one-third of the teachers men¬ 
tioned this, as exemplified by RK’s comment: “Creativity can be a challenge 
for me. I often find it difficult to come up with a good idea from scratch; I 
need some outside input to work with” (pre-survey). 

Thus, the primary concerns of teachers before they began the project were 
envisioning the concept behind the digital games and creating a final game 
that would be fun and interesting for the students to play. 

Designing and building their own games changed teachers’ perceptions 
of the challenges they would experience. When asked about the actual 
challenges they experienced in the post-survey, only 3 of the 21 teachers 
mentioned making the game fun and interesting, and only one mentioned 
creativity. The number of teachers concerned with creativity and making a 
fun and interesting game greatly decreased after the project, suggesting that 
the expected challenges had been exaggerated. 

In the post-survey, a challenge that a majority of the teachers discussed 
was programming. Many experienced difficulty in translating their design 
ideas to a game. Some teachers had to completely change their game designs 
to fit the program they used. For example, participant MG felt “limited by 
what kind of interface we choose for our game—our ideas can be really great 
and seem simple, but are hard to put into action” (post-survey). Partici¬ 
pant AA stated, “I wanted to make a story-based kind of game, but was too 
difficult. So I had to change the game ... [it] became a simple question and 
answer game with scoring points” (post-survey). 

Analysis of the games confirmed this challenge. For instance, 

Shellshocked 2 is a game that teachers DC and JH developed. The initial plan 
had players answering mathematics questions while venturing through the 
game space. But they were unable to accomplish that goal. In the final game, 
players could accidently access a part of the game that is only supposed to be 
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open after players finish a puzzle. Thus, one could complete the whole game 
without answering any mathematics problem. 

Problem Solving 

Teacher perceptions of the importance of problem solving in game design 
changed through the process of designing their own games. In the pre-sur¬ 
vey, the most commonly considered benefit was that the game design experi¬ 
ence would push students to gain a deep understanding of the mathematics 
content in order to design their own games, as reflected in these comments: 

When a kid designs an educational game he must understand the under¬ 
lying educational concepts associated with the game. (DC, pre-survey) 

This ‘learning through designing’ would benefit the students by prompt¬ 
ing them to consider their understanding of mathematics from a different 
point of view (‘how can I use what I have learned? rather than ‘here is 
what I have learned’), and providing them with a unique and engaging 
opportunity to create meaning truly for themselves. (LW, pre-survey) 


Although it is an important skill in mathematics learning, problem solv¬ 
ing was mentioned by only 2 out of the 18 teachers as a benefit of learning 
through game design in the pre-survey. Participant KG stated, “I also think 
that even the general thinking patterns required to make a game can help 
the students with their math skills (this would involve counting, patterns 
possible, time management, etc.)” (pre-survey). 

There was a marked increase in the number of teachers who mentioned 
problem solving as a benefit for students designing their own games in 
the post-surveys. In fact, close to 40% of the teachers discussed how game 
design provided opportunities for learners to exercise their problem-solving 
capabilities, as demonstrated by the following comments: 

Programming and coding also requires students to be effective problem 
solvers in order to get their games to work the way they want it to. (PH, 
post-survey) 

I think it would be beneficial to have students try to design games be¬ 
cause: Their problem-solving skills would be developed, which is incred¬ 
ibly valuable in all areas of school and life in general. (DC, post-survey) 

Pedagogical and Cognitive Skills 

What are the pedagogical and cognitive skills involved in teacher designed 
games? Analysis of data showed that teachers were already meeting expecta¬ 
tions for incorporating educational components in their games. Further, a 
link between a game’s score on the pedagogical rubric and on the cognitive 
rubric was revealed. 
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Pedagogical consideration. For the most part, teacher-designed games 
were meeting or exceeding expectations in the majority of the educational 
areas measured by the pedagogical rubric. More than half of the games 
either met expectations or exceeded expectations for six out of the nine cate¬ 
gories. These six categories were: representation, active learning, exploration 
and reasoning, engagement/motivation, participation, and user friendly and 
ease of play. Specifically, in 12 of the 14 games, students needed to represent 
mathematical ideas in a variety of forms and apply different concepts across 
different mathematical areas. For the categories of problem solving and 
strategy, half of the games either met expectations or exceeded expectations. 

Further, every game met or exceeded the expectation in the category of 
engagement/motivation. In addition, 13 of the 14 games were designed to 
encourage the majority of students to play and enjoy the games (participa¬ 
tion), and 11 of the games were user friendly. Therefore, we could conclude 
with confidence that teachers made reasonably enjoyable games. 

Connections was the only category that more than half of the games 
failed. Yet four games focused entirely on applications of mathematical 
ideas to the real world, therefore making connections their main focus. For 
example, The Date Game was about either a boy or a girl (the player decides) 
getting ready for a date by getting his or her hair done, buying a new outfit, 
and getting a present for the date. Through applications, the game connected 
solving linear equations to students’ lives. 

In summary, all but one pedagogical category was significantly addressed 
by at least half of the games designed by the teachers. Such integration of the 
important pedagogical knowledge into the games demonstrated that a major¬ 
ity of the teachers had deep understanding of these pedagogical principles. 

Cognitive processes. We also evaluated the games focusing on the cogni¬ 
tive aspect, which contained six categories in ascending order: remember, 
understand, apply, analyze, evaluate, and create. The first two categories were 
considered lower order thinking and the rest considered as higher-order 
thinking. The vast majority of the games (12 out of 14) required at least the 
use of understanding, while almost half of the games (6) required at least 
the use of applying to complete the game. That is, about half of the games 
embedded higher-order thinking skills. 

Most of the games (11) had cognitive scores between 2 and 4 (inclusive), 
indicating that most games involved understanding, applying, and analyz¬ 
ing. Only one game, Shellshocked 2, went beyond analyzing. Therefore, it 
appears that teachers were mostly having students use, at most, the skill of 
analyzing in the problems they had designed for their games. 

Having said that, when the teachers actively stepped away from the 
standard view of how to present mathematical problems (i.e., the work¬ 
sheet mentality), the types of problems they developed were very rich. 
Shellshocked 2 and Equalia exemplified the kinds of games created that 
embraced the environment of a digital game and posed rich problems. 
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One prevalent theme that arose was that the problems presented in many 
games were not much different from worksheets. The mathematical ques¬ 
tions were add-ons to the games instead of embodiments of the games. For 
example, Castle Quest was about a young wanderer who needed to free a 
castle from a curse by finding objects and manipulating algebraic expres¬ 
sions. Although it is a very engaging fantasy game with motivating story¬ 
lines, the math problems presented were disappointing. The math solution 
method is known in advance, and the problems do not enable players to 
develop new mathematical understanding. Further, the questions were not 
embedded in the storylines, making the math learning superficial. 

Link between pedagogical and cognitive skills. We identified a link be¬ 
tween the pedagogical components and the cognitive processes in the 
games. That is, a high score of a game on the pedagogical rubric tended to 
achieve a high score on the cognitive rubric. Half of the games received a 
total score of at least 8 on the pedagogical rubric, meaning that half of the 
games were either at least meeting expectations or close to doing so. Ad¬ 
ditionally, half of the games received a score of at least 2.45 out of 6 on the 
cognitive rubric, suggesting that at least half of the games were using at least 
the cognitive skill of understanding. 

Comparing the pedagogical rubrics with the cognitive scores, all the 
games that were above the median total score for the pedagogical rubric also 
had cognitive scores above the median. All games except one that had peda¬ 
gogical scores below the median also had cognitive scores below the median. 
The exception was Fraction Mahjong, which scored high on the cognitive 
rubric but low on pedagogical rubrics. Of those games that had exactly 8 for 
their pedagogical scores, two were below the cognitive median and two were 
above. 

At first glance, it seemed obvious that the pedagogy scores and the cogni¬ 
tive scores were connected. Upon further scrutinizing, however, we noted that 
the two rubrics focused on different components of the game. The pedagogi¬ 
cal rubric focused primarily on the mathematical contents posed, whereas 
the cognitive rubric looked at the problems needed to complete the game as a 
whole. For example, Fraction Mahjong scored low on the pedagogical rubric 
because the mathematical problems were not reflecting good mathematical 
thinking, but the strategy needed to complete the game required higher-order 
thinking, resulting in a high cognitive score. The link demonstrated that the 
teachers who were designing good mathematical questions were also having 
players use higher cognitive skills to complete the game. 

21 st Century Skills 

The analysis of the follow-up interviews suggested that the game design 
process enabled teachers to use all the 21 st century skills, and further scru¬ 
tiny of the games confirmed that they demonstrated all the 21 st century 
skills. First and foremost, teachers exercised creativity and innovation 
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skills in the development of their ideas for the games. Participant RK’s state¬ 
ment perfectly demonstrated this: “From nothing I made a game” (follow-up 
interview). 

They constantly exercised critical-thinking and problem-solving skills, 
most prominently during the process of programming and de-bugging the 
game, as exemplified by the following comment: “Problem solving was a 
large component of the design. This was used in figuring out how the pro¬ 
gramming language worked and the debugging” (CV, follow-up interview). 

The teachers practiced communication skills during all phases of the 
game design and development, especially during the process of creating the 
game scenarios and rules. For example, Shellshocked 2 and Fraction Town 
began with clear and concise instructions of what the players needed to do. 
Teachers needed to carefully consider the best ways to communicate their 
ideas to the players. 

Although not required, almost all the teachers collaborated in one way or 
another. For those who chose to work in groups, collaboration was an ongo¬ 
ing process, as described in the following comment: 

LW and I worked both collaboratively and parallel on the game. Initially 
we discussed our game vision and ideas, then we each played with Kodu 
separately to see what was possible. We reconvened and decided what 
would work and how we would implement it. We would pass the game 
back and forth between our computers to each try and problem solve/ 
debug separately and discuss the optimal/most effective solution. (CV, 
follow-up interview) 

Even for those who worked individually, they started their game designs 
in a small group setting and tested and discussed their prototypes during 
the class where classmates provided valuable feedback. The teachers often 
helped each other with programming even if they weren’t in the same group. 
For instance, in one class, teacher MT gave the class an impromptu lesson 
on Kodu. Many of the teachers stayed past class time to work with others. 
Needless to say, technology literacy was a significant component in building 
the digital games. All of the teachers produced built games and demonstrat¬ 
ed technological literacy in this regard. 

Some skills seemed unnecessary at the beginning, yet the data showed 
that they were exercised extensively. For example, when teachers had dif¬ 
ficulty programming, they researched previously completed games to get 
help: “I had to go to the Scratch website and find other creations that did 
what I wanted to do. Then I had to search for the right thing in the code that 
did what I wanted to do” (RK, follow-up interview). 

Here RK was using information literacy skills by finding, assessing, man¬ 
aging, and finally using previously developed Scratch games. Many of the 
teachers noted in class the use of tutorials and Internet searches as methods 
for learning how to program in their chosen game interface. 
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Flexibility and adaptability skills were also very apparent. When teachers 
could not program what they wanted, they adapted their plan to lit the pro¬ 
gramming platform they chose. For example, to create Awesome Adventure 
using Little Big Planet 2 (LBP2), the team’s original idea was to have players 
drop fractions that were written on blocks that they thought were equivalent 
to each other down a hole. The blocks with the correct answers would fill up 
the hole bit by bit until the players could cross it. If the answer was wrong, 
however, the block would fly out of the hole. However, the team could not 
program this in LBP2, so it decided to modify this idea by using triggers 
that could be flipped only if the right answer was given. For example, in 
one problem, the players had to find three fractions that were equivalent to 
different representations of fractions (decimal, percentage, visual). When all 
three correct blocks were put in the right space, an “and” switch was trig¬ 
gered, revealing a bumblebee. The player could then ride the bumblebee to 
the next level. The new design turned out to be more interesting for players, 
as it is more fun to fly on a bumblebee than just walk over blocks. 

All teachers demonstrated media literacy, as they all created media 
products—their games. Some teachers developed additional media prod¬ 
ucts to promote their games. One example was the movie trailer that the 
Call of Math team created to accompany and promote their game. Everyone 
demonstrated the life and career skills, because every team completed their 
games in an appropriate timeframe, and many worked collaboratively in 
small-group settings to create their game designs. 

Though not a specific skill as outlined by the Partnership for 21 st Century 
Skills, the crucial skill of perseverance is worthy of detailed discussion. For 
example, the Awesome Adventure team was confronted with a very dif¬ 
ficult problem: how to design questions in LBP2 so that players knew when 
they were right to continue. The team had to learn the various solutions by 
watching the tutorial videos that LBP2 offered and reading LBP2 level design 
forums on the Internet to overcome the difficulty of programming and solv¬ 
ing problems, demonstrating their determination. 

In the follow-up interviews and post-surveys, the theme of perseverance 
came up often. “I have learned to persevere. I had a lot of difficulty at the 
beginning, but kept at it and finally got a great product” (EM, post-survey). 
Programming could be frustrating and difficult, but it was rewarding when a 
finished game is made, as illustrated by this comment: 

Programming, for me, has this amazing ability to push me to seek solutions. 
Maybe it’s that there is this amazing feedback present where it smacks you 
in the face: ‘This does not work.’ So you persevere until you get to that mo¬ 
ment of completion. But at that moment of completion, you KNOW that it 
does what you want, there is this euphoria. (EL, follow-up interview) 

The game design and building experience allowed teachers the opportu¬ 
nity to complete a project that required a great deal of determination. 
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Discussion 

In this study, we sought to understand enactivism by applying the theory to 
practice and demonstrating a successful implementation of enactivism in 
a teacher education classroom. By adapting enactivist approaches, we have 
created a learning world that incorporates complex real-world problems 
while giving learners great freedom of exploration. 

Key Findings 

Among various positive outcomes, the most significant finding is that the 
enactivist learning world created rich opportunities for teachers to develop 
into problem solvers through perseverance. In typical classrooms, includ¬ 
ing preservice classrooms, students (in this case, preservice teachers) do not 
often show perseverance when solving a problem. Instead, they often give 
up relatively easily and wait for the correct answers, for various reasons. If 
one knows that the teacher will provide the answer at some point, she or he 
has no incentive to continue to struggle with the problem. Further, much 
schoolwork, such as assignments, is presented with clearly laid-out steps. 
There is no need for the student to understand the steps or how to solve the 
problem, but simply to follow the step-by-step procedure. All these push 
students, including preservice teachers, into becoming impatient problem 
solvers who lack initiative, perseverance, and retention and have an aversion 
to word problems (Meyer, 2010). 

In this enactivist world, we have intentionally provided rich opportunities 
for the teachers to explore freely, engendering self-initiative and eliminat¬ 
ing the crutches that give them excuses to give up. Digital game design, with 
no expert programmer or designer available, is a tool that embodies the 
skill of persistence. In this world, teachers have designed games in differ¬ 
ent programs that the instructor is not an expert in. Thus, one of the biggest 
crutches in the classroom is eliminated: The instructor sometimes does not 
know the answer. No longer can students rely on someone else to provide 
answers. In addition, self-initiative is encompassed in the process. When 
teachers overcome the problems, the result is a functional game that they 
can share with others with excitement and pride. 

An important implication is that instructors should be comfortable not 
being an expert in every aspect of their instruction and to provide learners 
freedom to discover. In this study, allowing the teachers to struggle through 
their difficulties resulted in the discovery of their creativity and confidence. 

Teachers in this study demonstrate all the 21 st century skills through the 
game design and building experience. This is consistent with the previous 
conclusions of Sardone and Devlin-Scherer (2010), who demonstrated that 
the use of digital games in the classroom allows teachers to use 21 st century 
skills, particularly in the areas of learning and innovation. 

Teachers learning in such an enactivist world changed their perceptions. 
The teachers did not recognize the creative skills that they possessed at the 
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outset. The creative process of designing games forced them to move out of 
their comfort zones, demonstrating that they were capable of making fun 
and interesting games. Such realization of their creativity may consequently 
help teachers foster their own students’ creative thinking. 

Implications for Practice 

Learning in such an enactivist world has impacted teachers positively in 
various ways, from changed perception to the demonstrated 21 st century 
skills. From the enactivist perspective, it would be difficult to teach someone 
else a skill that was not currently embodied or experienced by the teacher. 

As such, providing teachers with experiences of exercising 21 st century skills 
may help them better facilitate these skills in their future classrooms. 

In this enactivist learning world, teachers are the agents of their own 
learning: They determine their own role in the process and choose their own 
way of interacting with the software, colleagues, and players. One of the few 
requirements is the games’ focus on algebra. Choosing algebra is deliberate 
because its abstract nature often challenges teachers to relate it to students, 
which often makes it a boring topic to learn and difficult to teach. Focusing 
on algebra, teachers were forced to be creative in both the delivery format 
(digital game) and the problems. By experiencing this challenge, teachers be¬ 
came better equipped to challenge students’ preconceived notions of algebra. 

As mentioned earlier in this paper, in the 1980s, when the software 
market was not yet developed, many studies explored letting teachers create 
their own software. This effort failed because teachers lacked the needed 
skills and the time to develop effective software. Our study is a significant 
departure from that failed line of research for several reasons. First, the 
technology has advanced drastically in the last two decades. Web 2.0 tools 
provide user-friendly and easy-access means for anyone to create, share, and 
communicate beyond the walls of the classroom. Second, unlike teachers 
three decades ago, the current generation of preservice teachers have grown 
up in a digital world. This means that they are already familiar with social 
media, games, and many other tools. Finally, and most important, our focus 
is not on developing effective software but rather on the teacher’s experience 
of pedagogical and technical skills to promote 21 st century skills. Pedagogical 
and content learning is interwoven organically with game design. This shift 
of focus suggests that teachers spend time on pedagogical learning instead of 
polishing the quality of the game. 

This study shows that teachers require support to make the leap from 
worksheets to richer problems and problems that connect to the real world. 
Having teachers develop their own games is a positive move. Additional 
guidance in developing rich problems and using these problems in the class¬ 
room would further benefit teachers in this area. 

As discussed above, the teachers were immersed in game design and 
building—an inherently messy process. This process helped and even forced 
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them to take initiatives, to persevere, and to be creative in finding work¬ 
able solutions. That said, several teachers did become frustrated when they 
had trouble programming their games. Two important strategies proved to 
be effective. First, collaboration in various ways was highly recommended. 
Teachers were allowed to work in small groups and were encouraged to 
help each other within their own groups and across teams. Second, upon 
observing the negative feeling among a few teachers, the instructor designed 
a class activity exploring the pros and cons of this project. Interweaving 
pedagogical discussion with game design, the instructor asked the teachers 
to generate their own lists of advantages and disadvantages of game design 
in mathematics learning. Because the ideas emerged from their own con¬ 
versations rather than being imposed on them, the teachers had a deeper 
understanding of the value of the project and became more receptive to 
continuing the work. 

The final celebration showcasing their games was the culminating point, 
when everyone in the class was so excited and felt proud of him/herself. “We 
are so glad that we persevered!” was a reflective comment made by many of 
these teachers. 

Limitations and Recommendations for Further Research 

A limitation of the study is that the sample size for the follow-up inter¬ 
views was small, with only six teachers. Future study can include how the 
process of designing and building affects preservice teachers’ teaching 
style when they become inservice teachers. Researchers may also want to 
investigate whether the concept of learning by doing permeates into other 
areas. That is, do they have students learn by doing in areas other than 
digital games? 

Another limitation is that some subscales are based on a subjective expe¬ 
rience. One example is the engagement/motivation subscale. One may find 
a game interesting and engaging, whereas another could find it boring and 
dull. Future research is recommended to have the games’ target audiences 
(e.g., middle/high school students) evaluate games, which may provide a 
more definitive result. 
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Appendix A 


Pedagogical Rubric 


Category 

Minimal Representation 

Fair Representation 

Extensive Representation 

Problem solving 

No events or one small event that 
showed problem solving. 

One substantial event, or two or 
three distinct small events that 
showed problem solving. 

More than one substantial 
event, or more than three dis¬ 
tinct small events that showed 
problem solving. 

Active learning 

No events or one small event that 
allowed for active learning. 

One substantial event, or two or 
three distinct small events that 
allowed for active learning. 

More than one substantial 
event, or more than three dis¬ 
tinct small events that allowed 
for active learning. 

Exploration and 
Reasoning 

No events that allowed for explora¬ 
tion and reasoning. 

One substantial event, or two 
or three distinct small events 
that allowed for exploration and 
reasoning 

More than one substantial 
event, or more than three dis¬ 
tinct small events that allowed 
for exploration and reasoning. 

Connections 

No events or one small event that 

showed connections. 

One substantial event, or two or 
three distinct small events that 

showed connections. 

More than one substantial 
event, or more than three dis¬ 
tinct small events that showed 

connections. 

Strategy 

No events or one small event that 
showed strategy. 

One substantial event, or two or 
three distinct small events that 
showed strategy. 

More than one substantial 
event, or more than three dis¬ 
tinct small events that showed 
strategy. 

Participation 

Does not encourage most players 
to participate. 

Encourages most players to 
participate. 

Encourages all players to 
participate. 

Engagement/ 

Motivation 

The game is not interesting or fun 
to play. 

The game is interesting and fun 
to play. 

The game is really interesting 
and really fun to play. 

User Friendly 
and Ease of Play 

Confusing or unclear objectives 
or instructions. Many elements 
that caused major frustration in 
play and may cause player to stop 
playing. 

Clear objectives and instructions 
of the game. A few elements that 
cause minor frustration in play. 

Very clear objectives and 
instructions of the game. No 
elements that cause frustration. 

Collaborations 

The game doesn’t allow any form 
of collaborations with other players 
or with other objects in the game. 

The game allows for some col¬ 
laboration. 

The game encourages several 
collaborations, whether it is 
with non-player characters or 
with other gamers. 

Scaffolding 

No scaffolding occurs within the 
game. There is no support for pro¬ 
gression of knowledge or concepts 
in the game. 

The game creates an adequate 
platform for scaffolding through 
tutorials or guides. 

The game goes above and 
beyond in setting up stages 
and levels that progress the 
concepts conveyed in an 
increasingly challenging way. 

Assessment 

The game has no characteristics 
that help the gamer to assess their 
level or situation within the game. 
Gamer may feel lost when trying to 
understand his/her abilities and/or 
achievement in the game. 

The game provides tools (e.g. 
hit points, level ups, gauges/ 
meters, visual maps, messages 
and alerts) to adequately assess 
the gamers progression through 
the game. 

The game is set up in a way 
that makes the gamer feel as 
though he/she knows how his/ 
her character is doing, what 
levels have been achieved or 
need to be, and is able to make 
conjectures on the gameplay 
because of it. 
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Definitions of Categories 

This rubric is designed to see if a game was created that promotes good 
teaching and good learning. That is, the game isn’t just a digital work¬ 
sheet. Instead the game promotes inquiry, critical thinking, and explo¬ 
ration. In no way would we suggest that these categories are mutually 
exclusive. 

Problem solving. Problem solving “means engaging in a task for which 
the solution method is not known in advance. In order to find a solution, 
students must draw on their knowledge, and through this process, they 
will often develop new mathematical understanding” (NCTM, p. 52). 

The game should encourage students to “build new mathematical knowl¬ 
edge through problem solving; solve problems that arise in mathematics 
and in other contexts; apply and adapt a variety of appropriate strategies 
to solve problems” (p. 52). 

Representation. Representation refers “both to process and to prod¬ 
uct—in other words, to the act of capturing a mathematical concept or 
relationship in some form and to the form itself” (NCTM, p. 67). The 
game should encourage players to “create and use representations to 
organize, record, and communicate mathematical ideas; select, apply, 
and translate among mathematical representations to solve problems; use 
representations to model and interpret physical, social, and mathemati¬ 
cal phenomena” (NCTM, p. 67). (Note: If the game gives players an act 
of representation that is repeated throughout the game, then this is one 
substantial event. For example, if students are expected to convert frac¬ 
tions to another form for the whole game, that is one substantial event 
for representation.) 

Active learning. Here we are looking for components of the game that 
make a student think about what they are doing. As a negative definition, 
inactive learning occurs when a student is just given a simple question 
and is asked to answer it. Instead active learning occurs when stu¬ 
dents develop their own questions and/or are presented with a complex 
enough question that multiple steps are required to answer it (and these 
steps are not laid out for the student). Players may have to think outside 
of the game. The mathematics is not necessarily explicit. 

Exploration and reasoning. Rich problems allow for students to explore 
and discover the solution rather than solving it by straightforward com¬ 
putations. They allow for students to “make and investigate mathemati¬ 
cal conjectures” (NCTM, p. 56) and to look for and identify patterns 
and structures. Here we are looking for problems that allow students to 
further their understanding of the topic by delving into the problem and 
justifying their solution. 

Connections. Here we are looking for components of the game that 
relate the material to different areas in mathematics (e.g., if the game is 
specifically examining fractions, then if the game also examines linear 
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functions, this would be an example of a connection to a different area 
in mathematics), different subject areas (e.g., the game includes compo¬ 
nents related to physics or biology), or real-world experiences that the 
students whom the game is targeted at would be knowledgeable of and 
experience in their own lives (e.g., relating percent to tax). 

Strategy. Within the game, do players have to strategize to arrive at 
the end? That is, there are elements of the game that require the player 
to make a plan so that they can accomplish a component of game play. 
For example, in the Portal games, you have to plan where you are going 
in order to place your portals so that you can finish a level. This cat¬ 
egory focuses on game play. 

Participation. A game is well designed if both strong and weak students 
want to play it. The educational components of the game are set up in a 
way that does not exclude weak students or bore strong students. That 
is, the problems are rich enough to allow students of all levels the op¬ 
portunity to attempt the problems. This category focuses on the prob¬ 
lems posed. 

Engagement/motivation. Game elements interest students. Examples 
include (but are not limited to) an interesting narrative, working toward 
a goal, competition (with self or others), and increasing level of diffi¬ 
culty. This category focuses on game play. 

User friendliness and ease of play. Here we are looking at whether the 
game was frustrating or not. Did you understand what your goals and 
objectives were? If you were told to accomplish a task, were clear instruc¬ 
tions given on how that could be done? Is the level of the game play at a 
reasonable level? 

Definitions for Rubric 

Event. This is any component of the game. It can include explicit or im¬ 
plicit problems presented, strategies needed to solve the game, or anything 
else that is a feature of the game. 

Substantial event. This is a component of the game that would either 
take substantial game play time to do (20-50% of game time) or is a 
primary component of the game (e.g., a theme that runs throughout the 
game but doesn’t necessarily take up a lot of game play). 

Distinct events. These can include the same type of problem but in dif¬ 
ferent contexts. For example, if all of the questions are solving real-world 
problems (e.g., applying percentage), but each of the situations is differ¬ 
ent (e.g., adding tax, determining how much is saved, determining new 
price), then these are considered distinct events. 

If the game involves solving one very rich problem, we take that to be 
more than one substantial event. 
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Appendix B 

Cognitive Rubric 

To evaluate the type of knowledge needed to complete these games, we will use 
the following table taken from “A Model of Learning Objectives” by the Centre 
for Excellence in Learning and Teaching at Iowa State University (2011). 


Table A. The Cognitive Processes Dimension: Categories, Cognitive Processes (and Alternative Names) 

Lower-Order Thinking Skills Higher-Order Thinking Skills 


Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Recognizing 

(identifying) 

Interpreting 

(clarifying, 

paraphrasing, 

representing, 

translating) 

Executing 
(carrying out) 

Differentiating 

(discriminating, 

distinguishing, 

focusing, 

selecting) 

Checking 

(coordinating, 

detecting, 

monitoring, 

testing) 

Generating 

(hypothesiz¬ 

ing) 

Recalling 

(retrieving) 

Exemplifying 

(illustrating, 

instantiating) 

Implementing 

(using) 

Organizing 

(finding 

coherence, 

integrating, 

outlining, 

parsing, 

structuring) 

Critiquing 

(judging) 

Planning 

(designing) 


Classifying 

(categorizing, 


Attributing 

(deconstructing) 


Producing 

(construct) 


subsuming) 

Summarizing 

(abstracting, 

subsuming) 

Inferring (concluding, 
extrapolating, 
interpolating, 
predicting) 

Comparing 
(contrasting, 
mapping, matching) 

Explaining 

(constructing models) 

Note. Table 1 adapted from Anderson & Krathwohl, 2001, pp. 67-68. 

It is our belief that each of the above knowledge types is nested within each 
other. Like a matryoshka doll or Russian nesting doll, the knowledge types 
on the left are part of the knowledge types above it. That is, we cannot apply 
knowledge without first remembering and understanding the knowledge. 

For each game, we determine how many problems exist in the game. 
Then each problem will be given a score from 1 to 6, where 1 means that 
the most knowledge that the player needs to apply for that problem is 
remembering, and 6 means that the most knowledge that the player needs 
to apply for that problem is creating. If 6 is given, that also means that 
the player has also used all of the other knowledge types to complete the 
problem. 
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Once each of the problems has been given a score, the mean of the problems 
is the score for that game. For example, if a game has three problems in which 
two problems apply knowledge (score of 3 each) and one problem analyzes 
knowledge (score of 4), then the overall score for the game is 3.3. 
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